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I ABSTRACT 

| A follow-up was made to a feasibility study of the 

I use of computer assisted instruction in teaching basic electronics, 

| The study was designed to obtain additional empirical data, collected 

| in a real time training environment, on the effectiveness of CAI as a 

| teaching method re lati ve to the con v entional mode of instruction 

I (CI) . Four specific objectives were set forth: (1) a comparison of 

I the CAI group versus the CI group on two performance cr it eria— -test 

\ achievement and time to complete course material; (2) a comparison of 

I the two groups on four follow-up performance 

j criteria — written/performance tests and setback/failure rates; (3) an 

!' assessment of student attitudes toward CAI; and (4) a comparison of 

l the revised versus the feasibility study CAI material.. The findings 

\ completely support the basic conclusion drawn in the feasibility 

\ study that CAI is as effective as CI in teaching basic electronics 

| and further demonstrates the capability of CAI to reduce training 

[ time to a significant degree, (Author/JY) 



s US CONTINENTAL ARMY COMMAND 
£ COMPUTER ASSISTED INSTRUCTION PROJECT 

UJ 



THE IMPLEMENTATION OF COMPUTER ASSISTED INSTRUCTION 
IN US ARMY BASIC ELECTRONICS TRAINING 



FOLLOW UP OF A FEASIBILITY STUDY 



TECHNICAL REPORT 69 1 



SEPTEMBER 1969 




O' 

r- 

o 

o 





I 



< 




US ARMY 
FORT 



SIGNAL CENTER AND SCHOOL 
MONMOUTH, NEW JERSEY 



% 







EDO 38021 



THE IMPLEMENTATION OF COMPUTER ASSISTED INSTRUCTION 
IN US ARMY BASIC ELECTRONICS TRAINING 



U.S. DEPARTMENT OF HEALTH. EOUCATION FOLLOW-UP OF A FEASIBILITY STUDY 

* WELFARE 
OFFICE OF EDUCATION 
THIS OOCUMENT HAS BEEN REPROOUCEO 
EXACTLY AS RECEIVEO FROM THE PERSON OR 
ORGANIZATION ORIGINATING IT. POINTS OF 
VIEW OR OPINIONS STATEO 00 NOT NECES- 
SARILY REPRESENT OFFICIAL OFFICE OF EOU- 
CATION POSITION OR POLICY 



Alexander A. Longo 



September 1969 



Technical Report 69-1 



Approved by 



Dr, Vincent P, Cieri 
Technical Director 



COL T, F, Schweitzer 
Project Manager 



The findings in this report are not to be construed as 
an official Department of the Army position, unless so 
designated by other authorized documents. 



CAI Project 

US Army Signal Center and School 
Fort Monmouth, New Jersey 07703 



FOREWORD 



The conduct of a study such as this typically involves the concerted 
effort of a large number of personnel operating across varied departments 
and disciplines. It is no exaggeration that the success of CAI at USASCS 
thus far has been due to the complete dedication of all the individuals 
associated with this project. The contribution of the Course Development 
and Programming/ System Operations Divisions within the CAI Project are 
particularly acknowledged. The assistance of the Academic Records Divi- 
sion of the Office of Secretary, the Radar Division and Examination Branch 
of the Department of Specialist Training and the Training Aids Division of 
the Office of Logistics at USASCS are all worthy of special note. The 
fruitful interaction of all the above departments is itself an achieve- 
ment shared by all. If the past is any indication, the future promises 
a breakthrough in educational technology through the use of computer 
assisted instruction. 



Vincent P. Cieri 
Technical Director 
CAI Project 

US Army Signal Center and School 
Fort Monmouth, New Jersey 07703 



BRIEF 



This is the first in a series of studies designed to assess the 
effectiveness of implementing computer assisted instruction (CAI) in the 
US Army Basic Electronics training. The present study represents a logi- 
cal follow-up to the feasibility study on computer assisted instruction 
in US Army Signal Center and School (USASCS) . It approximates a replica- 
tion of the feasibility study employing a revised CAI program with a much 
larger sampling of the student population entering the Common Subjects 
Branch of Basic Electronics at USASCS. The overall objective was to obtain 
additional empirical data, collected in a real time training environment, 
on the effectivenss of CAI as a teaching method relative to the current 
conventional mode of instruction (Cl). 

The paradigm for this study was set forth in four specific objectives: 
(a) a comparison of the CAI group versus the Cl group on 2 performance 
criteria: test achievement and time to complete course material; (b) a 

comparison of the CAI group versus the Cl group on 4 follow-up performance 
criteria: written/performance tests and setback/failure rates; (c) an 

assessment of student attitudes toward CAI; and (d) a comparison of the 
revised versus the feasibility study CAI material. An equivalent groups 
(matching by pairs) design was employed. Two matched groups with an N of 
278 each were selected for evaluating the comparative effectiveness of 
CAI and Cl. The experimental setting was the first 2 weeks of basic elec- 
tronics at USASCS: Week 1 was taught separately by CAI and Cl methods; 
Week 2, by the Cl method alone. 

The results comparing the CAI and Cl groups on achievement and time 
to complete the course material indicated that the two groups were equiv- 
alent with respect to their achievement but the CAI group completed their 
prescribed training (Week 1) in significantly less time. In comparison 
with the fixed time of 11 1/4 hours of instruction for the Cl group, the 
CAI group's mean time of 8.99 hours represented a 20.1% reduction in 
training time. Likewise, both groups demonstrated equivalent performance 
on all 4 follow-up criteria obtained at the end of Week 2 of basic elec- 
tronics training: written/performance tests and incidence of setbacks/ 

failures. The CAI group maintained a favorable disposition toward CAI 
over an interval of 2 weeks of instruction, despite their taking Cl during 
the second week of training. The data also illustrated the important role 
which course revision has in the development and application of CAI. The 
overall reduction of training time of 20.1% was judged to be not only 
operationally and administratively but statistically significant as well. 

The findings completely support the basic conclusion drawn in the 
feasibility study that CAI is as effective as Cl in teaching basic elec- 
tronics and further demonstrates the capability of CAI to reduce training 
time to a significant degree. 
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The Implementation of Computer Assisted Instruction 
in US Army Basic Electronics Training 

Follow-up of a Feasibility Study 



This study is a follow-up of the feasibility study^ on computer 
assisted instruction (CAI) in US Army Basic Electronics training conducted 
at the United States Army Signal Center and School (USASCS), Fort Monmouth, 
N. J. The overall purpose of the feasibility study was to provide the 
United States Continental Army Command objective evidence to determine the 
effectiveness of CAI as a medium in teaching basic electronics. It pro- 
vided a comparison of student performance on 11 1/4 hours of basic elec- 
tronics using CAI with equated groups taking the same subject matter under 
more conventional methods of instruction ( ins true tor/ TV) . The results, in 
brief, indicated CAI to be effective, efficient and applicable for training 
students in basic electronics. Due to certain unavoidable constraints, how- 
ever, the feasibility report emphasized the sampling limitations inherent in 
the study regarding both the number of subjects and the amount of material 
employed. Therefore, follow-up research such as the current evaluation is 
indicated to determine if increased confidence in the conclusions drawn in 
the feasibility study can be made. 

The present study approximates a replication of the feasibility study 
employing a revised CAI program with a much larger sampling of the student 
population entering the Common Subjects Branch of Basic Electronics at 
USASCS. It provides additional empirical data, collected in a real time 
training environment, on the effectiveness of CAI as a teaching method 
relative to the current conventional mode of instruction (Cl). A compari- 
son between current CAI findings and those reported in the feasibility 
study is also provided. Included is evidence on achievement, time to 
complete course material and attitudes toward CAI. 

CAI Program Description 

The CAI program developed in the feasibility study represents the 
first 11 1/4 hours of Basic Electronics training taught at USASCS. At its 
inception, special attention was given to insure that the CAI course 
material was equivalent in content to that presented in the conventional 
classroom. Thus, the CAI material was developed to meet the terminal per- 
formance objectives of conventional training and, when completed, the CAI 



International Business Machines Corp. A Feasibility Study of Com- 
puter Assisted Instruction in U. S. Army Basic Electronics Training. Final 
Report. Contract Nr. DAAB07-67-C-0578. Gaithersburg, Md. Feb, 1968. 
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course lessons were reviewed by Signal School personnel for content equiv- 
alence. 

The material selected is normally taught on Thursday and Friday of the 
first week in the Common Subjects Branch of Basic Electronics. It includes 
the material in the lesson plan set 280.0-1-LP (25-38), dated Jan 1967, 
covering the following topics: 

a. Introduction to Electricity 

b. Care and Use of the Multimeter 

c. Batteries 

d. Resistors 

e. Resistor Applications 

In order to adapt the course material to CAI, the content of these topics 
was organized into 4 subject matter areas called course segments. These 
basically consist of a pretest, a series of lessons and a lesson test. 
(Further details pertinent to the development of the CAI material: i.e., 

course logic, instructional frame logic, etc., are reported in the feasi- 
bility study.) 

This CAI program was administered to 18 representative students just 
entering basic electronics training. Analysis of the student performance 
records indicated the need for certain minor revisions in the CAI program. 
Figure 1 indicates the extent of these revisions. Most of the revision 
was concentrated in Segment I. This consisted in a consolidation of six 
lessons into two lessons. The purpose was to convert the theoretical ori- 
entation of this segment into more practical information on electricity. 

Tills modified CAI program comprised the training material administered to 
the CAI students during Week 1 of the follow-up study. 1 



Method 

1. Sampling. The sample for this study was selected from the normal in- 
puts of draftees and Regular Army students to the Common Subjects Branch 
of Basic Electronics. In order to obtain a representative sample of this 
population, students were selected from a number of Military Occupational 
Specialties (MOS) within 2 different Army' career groups: Radar /Microwave 

Maintenance and Fixed Plant Communications Equipment Maintenance. Both 
the MOS and student selections were randomly obtained. 

a. Student selection: Subsequent to their arrival for training, a 

roster of students was compiled by the Signal School Registrar. This 
(initial class) roster categorizes the student inputs by MOS and lists 



The CAI program was executed by the IBM 1500 Instructional System. 
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them by name according to their arrival time, which for all practical pur- 
poses is considered to be random. The first 20 students of a predesignated 
MOS were selected from this roster and assigned to a CAI class. The re- 
maining students were assigned routinely to conventional instruction classes, 

b, MOS selection: The type, number and input size of the MOS's in the 

2 stated career groups convening for electronics training at USASCS varies 
weekly. These factors determined the breadth of sampling possible in the 2 
given MOS series. Within these operating limits, the specific MOS from which 
students were drawn weekly for CAI training was selected on the basis of a 
table of random numbers. This student-MOS selection process was repeated for 
a period of 15 weeks. 

2. Training-Testing Paradigm. During the course of the study (Oct. 1968 - 
May 1969} the 2 separate groups of incoming students reported to their re- 
spective mode of instruction (CAI/CI), Student performance (achievement and 
time to complete training) and attitude during the first 2 weeks of basic 
electronics (Phase I) comprised the extent of the comparative evaluation of 
these groups. Since new students require an orientation regarding their MOS, 
shop practices, etc., the actual extent of basic electronics training in 
Week 1 only amounted to 11 1/4 hours. The first week of basic electronics 
instruction was taught with both CAI and Cl methods and represented the only 
major source of experimental variation between the 2 groups. Both groups 
were taught Week 2 of Phase I via conventional methods alone. 

a. Performance criteria: On Friday of the first week of instruction, 

each group was administered an 85 item test designed to measure their 
achievement. This criterion measure was essentially the same test used in 
the feasibility study except for minor revision of several items. The items 
were selected on the basis of 2 general criteria: (1) an average item dif- 
ficulty level of .65, and (2) a minimum discrimination index of .20. To 
increase the test's "floor" and "ceiling" a few very easy and difficult items 
were also included. The average difficulty level obtained was .67. The 
test's validity was insured, by tailoring items to the course objectives and 
submitting them to the Department of Specialist Training, USASCS, for review 
and concurrence. Two measures of reliability were obtained: (1) „90 (split- 

half); and, (2) .87 (Kuder-Richardson) . Likewise, on Friday of the second 
week of instruction, both groups were administered a phase examination 
(written and performance) covering the first 2 weeks of instruction (48 
hours). Two other performance criteria were obtained directly from school 
records: the incidence of setbacks/failures. These 4 measures represented 

follow-up criteria on the relative impact of the variation in training 
method during Week 1. 

The amount of time for the CAI and Cl groups to complete their re- 
spective course material was designated as a second criterion of performance. 
Typically, the Cl method was geared to a fixed learning schedule for all 
students (11 1/4 hours), whereas the CAI method provided a capability for 
variable learning schedules contingent on the individual' s progress through 
the course. Calculation of the time to complete Week 1 material for the 
CAI students was automatically recorded in the student performance record 
generated by the computer. 
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b. Attitude measure: The CAI group was also administered an attitude 

questionnaire composed of 22 Likert items designed to assess their overall 
opinion toward CAI* The questionnaire was constructed in 2 parts: (1) a 

comparison of CAI with Cl (11 items); and, (2) an assessment of CAI alone: 
e.g., hardware, software, etc. (11 items). An ordinal scale of 1-5 for 
each item yielded total score variations ranging from pro-CAI - neutral - 
pro-CI for part I; and, "favorable" - neutral - "unfavorable" for part II. 

Since none of the incoming students had ever experienced CAI before, 
the situation-oriented nature of the items precluded any pretesting prior 
to CAI itself. Instead, a pre-CI vs. post-CI attitude paradigm was employed. 
Thus, essentially the same questionnaire was administered twice: (1) at the 

completion of CAI (Week I); and, (2) at the completion of Cl (Week II). 

Only minor modifications in the instructions to adapt the questionnaire to a 
new situation distinguished the separate administrations. The nature of the 
attitude instrument and the logic of the situation itself precluded the use 
of other pre-post testing combinations of experimental and control groups 
as suggested by Campbell. 

3. Matching. In order to circumvent many of the administrative problems 
associated with the conduct of a matched groups experiment in an ongoing 
training environment, an "after-the-fact" matching technique was utilized. (3) 
The label "after-the-fact" signifies that matching of students is effected 
subsequent to their assignment to the different modes of instruction (not 
prior to it) on the basis of a matching variable already available prior to 
the conduct of the study. Thus, as time permits, matching is accomplished 
either during or after the experimental treatments are administered, with- 
out any interference with the student's ongoing training schedule. 

In the present study, matching was performed on the basis of a pre- 
dicted Phase I score (i.e., expected achievement in the first 2 weeks of 
training). A multiple regression equation, based on an N of 1500 with 
R « .70, provided the predicted scores. In turn, the scores required by 
the regression equation were obtained from the student personnel files 
which they possess at the time of their arrival for training. As explained 
above, the availability of various aptitude scores prior to training by- 
passed the need for any pretesting, and consequently permitted matching 
of students after classes convened. It should be noted, that the chance 
to obtain perfect matching of the treatment groups is increased by this 
procedure because a much larger pool of control group students is made 
available. Thus, for every subject in the CAI pool there accumulated 
several potential counterparts in the Cl pool, which facilitated the 
matching process. The training of the prescribed 15 classes of students 
yielded 2 pools of subjects, one for each instructional method. The Cl 
pool contained about 5 times as many subjects as the CAI pool. From 
these 2 treatment pools, matched pairs of students were drawn for data 
analysis. Selection of a matched counterpart within a given score level 
was done on the basis of a table of random numbers. Thus, 2 matched groups 



